1. Field of the Invention
The present invention relates to a signal processing apparatus for processing pixel signals by a predetermined range, and more particularly, to a median filter for reducing noise included in input image data and the like.
2. Description of the Related Art
A median filter can be used to perform processing of setting a pixel region having a predetermined size around each pixel and replacing original data possessed by the pixel with a median of all data within the surrounding pixel region. The median filter is capable of removing an isolated point, such as spike noise, without impairing the resolution of image data, and is often used as a function of image processing software in a computer.
Recently, in accordance with an increasing demand for obtaining an image comparable to an image obtained by a silver-halide camera by increasing the number of pixels of a digital camera, the number of pixels of an image sensor has been increased, and signal processing for realizing higher picture quality has been requested. Digital cameras generally use a single image sensor having color filters from the viewpoint of reducing the size and the cost of the camera. Usually, if it is intended to obtain high resolution in an image, false colors caused by the arrangement of color filters are generated. If it is intended to prevent false colors, the resolution is degraded. Accordingly, in order to obtain a high-resolution image with accurate color reproduction, it is desired to provide a median filter in the form of hardware.
Japanese Patent Application Laid-Open (Kokai) No. 5-233804 (1993) relates to a conventional example of realization of a median filter in the form of hardware. FIGS. 18A-18G illustrate the flow of data processing of such a filter. FIG. 2 illustrates the relationship between respective data C1-C12 shown in FIGS. 18A-18G, and pixel positions. In the following description, the same reference numeral is used for a pixel and image data output form the pixel.
It is assumed that in an initial state, the signal levels of respective data C1-C9 in a pixel region surrounded by solid lines shown in FIG. 2 are ranked in the order of magnitude, and ranked pixels are arranged in the order of magnitude as shown in FIG. 18A. Although the target pixel at that time is a central pixel C5 in the image region surrounded by the solid lines, data C3 at the center of the arrangement of the data C1-C9 is output from the median filter as data for this target pixel after filtering processing, instead of the data C5.
If the target pixel shifts from the pixel C5 to the right by one, i.e., to a pixel C8, the image region to be processed by the median filter becomes a region including pixels C4-C12 surrounded by broken lines. First, processing of searching for data of a pixel C1 from among data that become unnecessary because they leave the new pixel region, and deleting the data from the group of ranked data is performed. Address data indicating the position within the pixel region is added to each data. For example, XY coordinates (0,0) in a system comprising remainders of 3 is added to the data C1. Hence, the data C1 is specified from the nine data C1-C9 utilizing the XY coordinates (0,0).
After searching for the data C1, eight data other than the data C1 are rearranged to provide a state in which eight data other than the data C1 are ranked as shown in FIG. 18B. Since only one data has been deleted, the order of magnitude of each of the eight data is either the same as before or an order lower than the previous order by one.
Next, the magnitude of data C10 from among three data newly added to the pixel region to be processed by the median filter is compared with the magnitudes of the eight ranked data to determine the order of magnitude of the data C10, and the nine data are rearranged in accordance with the new order. For example, if the value of the data C10 is between data C4 and C2, the order shown in FIG. 18C is obtained. At that time, since only one data is newly added, the order of each of the eight data is either the same as before or an order higher than the previous order by one.
Next, the data of a pixel C2 is searched for from among data that become unnecessary because they leave the new image region, and is deleted from the group of the ranked data. At that time, the search of the data C2 is performed utilizing xy coordinates (0,1) added to the data C2, and the data C2 is specified from among the nine data. Then, eight data other than the data C2 are rearranged to provide a state in which eight data other than the data C2 are ranked as shown in FIG. 18D. Next, the magnitude of data C11 from among the three data newly added to the pixel region to be processed by the median filter is compared with the magnitudes of the eight ranked data to determine the order of the data C11, and the nine data are rearranged in accordance with the new order. For example, if the value of the data C11 is between data C7 and C3, the order shown in FIG. 18E is obtained.
By searching for data of a pixel C3 in the same procedure, eight data other than the data C3 are rearranged as shown in FIG. 18F. After determining the order of data C12 by comparing the magnitude of the data C12 with the magnitudes of eight ranked data, the nine data are rearranged in accordance with the new order as shown in FIG. 18G.
Thus, data can be rearranged in the order of magnitude in the new pixel region C4-C12. In this case, the output of the median filter for the target pixel C8 in the pixel region is data C11 in the center of the order of magnitude of the signal levels of the data C4-C11.
According to the above-described approach, the order of magnitude of the group of data to be subjected to filtering processing when performing filtering processing of the immediately preceding target pixel can be utilized. Hence, it is possible to reduce the number of comparison operations than in a case in which the comparison of the magnitudes of all data to be subjected to filtering processing is newly performed for each target pixel, and therefore this approach is effective. Furthermore, since the removal of data and the addition of data relative to the group of ranked data are performed one by one, the movement of data is limited to an adjacent position. As a result, it is relatively easy to realize a median filter in the form of hardware.
However, in the above-described conventional approach, since the removal of unnecessary data, the calculation of the order of magnitude of new data, and the rearrangement of data are performed for each data, the number of operations of comparison and rearrangement of data increases, resulting in an increase in the processing time. Furthermore, as the size of the image region to be subjected to filtering processing increases, the signal processing time for one target pixel increases in proportion to the since of the image region. As a result, high-speed signal processing cannot be performed.
The present invention has been made in consideration of the above-described problems.
It is an object of the present invention to minimize the number of operations of comparison and rearrangement of data and increase the processing speed with a small circuit scale, when utilizing a median filter in the form of hardware.
One embodiment relates to a signal processing apparatus for setting a region having a predetermined size in order to process pixel signals in an image pickup device, shifting the set region, and performing signal processing based on pixel signals within each region by sequentially shifting the region. The apparatus includes a first new-data-order determination unit that determines an order of magnitude of levels of pixel signals to be newly subjected to signal processing as a result of shift of the region. A new-data rearrangement unit rearranges new pixel signals to be newly subjected to signal processing, based on the order of magnitude determined by the first new-data-order determination unit. A second new-data-order determination unit determines an order of magnitude of new pixel signals obtained by combining the new pixel signals rearranged by the new-data rearrangement unit and old pixel signals which have been subjected to signal processing before the shift of the region, and an order of magnitude of signal levels of which have been determined, by comparing levels of the new pixel signals rearranged by the new-data rearrangement means with the levels of the old pixel signals. An all-data-order determination unit determines an order of magnitude of signal levels of all pixels obtained by combining the new pixel signals and the old pixel signals, based on the order of magnitude of the new pixel signals determined by the second new-data-order determination unit. An all-data rearrangement unit rearranges signals of the all pixels, based on the order of magnitude of the signal levels of all the pixels determined by the all-data-order determination unit.
Another embodiment relates to a signal processing method for setting a region having a predetermined size in order to process pixel signals in an image pickup device, shifting the set region, and performing signal processing based on pixel signals within each region by sequentially shifting the region. The method includes a first new-data-order determination step of determining an order of magnitude of levels of pixel signals to be newly subjected to signal processing as a result of shift of the region, a new-data rearrangement step of rearranging new pixel signals to be newly subjected to signal processing, based on the order of magnitude determined in the first new-data-order determination step. A second new-data-order determination step determines an order of magnitude of new pixel signals obtained by combining the new pixel signals rearranged in the new-data rearrangement step and old pixel signals which have been subjected to signal processing before the shift of the region, and an order of magnitude of signal levels of which have been determined, by comparing levels of the new pixel signals rearranged in the new-data rearrangement step with the levels of the old pixel signals. An all-data-order determination step determines an order of magnitude of signal levels of all pixels obtained by combining the new pixel signals and the old pixel signals, based on the order of magnitude of the new pixel signals determined in the second new-data-order determination step. An all-data rearrangement step rearranges signals of all the pixels, based on the order of magnitude of the signal levels of all the pixels determined in the all-data-order determination step.
Yet another embodiment relates to a storage medium storing a program for a signal processing method for setting a region having a predetermined size in order to process pixel signals in an image pickup device, sequentially shifting the set region, and performing signal processing based on pixel signals within each region by sequentially shifting the region. The method includes a first new-data-order determination step of determining an order of magnitude of levels of pixel signals to be newly subjected to signal processing as a result of shift of the region. A new-data rearrangement step rearranges new pixel signals to be newly subjected to signal processing, based on the order of magnitude determined in the first new-data-order determination step. A second new-data-order determination step determines an order of magnitude of new pixel signals obtained by combining the new pixel signals rearranged in the new-data rearrangement step and old pixel signals which have been subjected to signal processing before the shift of the region, and an order of magnitude of signal levels of which have been determined, by comparing levels of the new pixel signals rearranged in the new-data rearrangement step with the levels of the old pixel signals. An all-data-order determination step determines an order of magnitude of signal levels of all pixels obtained by combining the new pixel signals and the old pixel signals, based on the order of magnitude of the new pixel signals determined in the second new-data-order determination step. An all-data rearrangement step rearranges signals of all the pixels, based on the order of magnitude of the signal levels of all the pixels determined in the all-data-order determination step.
The foregoing and other objects, advantages and features of the present invention will become more apparent from the following description of the preferred embodiments taken in conjunction with the accompanying drawings.